---
title: "Global Models with Modeltime"
author: "Alberto Almuiña"
date: '2021-08-02T02:13:14-05:00'
description: An introduction to global models with modeltime
slug: global-models
tags:
- modeltime
categories:
- Time Series
---
Modeltime 📈 and Global Models 🌏
Modeltime is a time-series focused package that has decided to focus on global models as the main strategy to meet the new scalability challenges that have been emerging in recent years. The number of time series has increased exponentially: organizations have more and more information at their fingertips and at a more disaggregated level*. Unfortunately, traditional approaches, such as ARIMA models, in which a model is calculated for each of the available time series, end up not being scalable (Suppose an organization with 10,000 products, it would need to iterate to create 10,000 models, and that is without taking into account hyperparameter tuning). This iterative strategy can become a real nightmare as organizations grow because it is not scalable.
What is the alternative proposed by Modeltime?
The alternative proposed by Modeltime is to use panel data and global models. What do these concepts mean? Panel data is basically a dataset containing several time series stacked on top of each other. An example can be seen in the following image:
A global model is a single model that forecasts all time series at once. Global models are highly scalable. An example is an XGBoost model, which can determine the relationships for all 1000 time series panels with a single model. Let’s look at an example:
Use Case: Prediction 🔮 of Electricity Generation 💡 by Sectors of the Generalitat de Catalunya.
⚠️ Warning️ ⚠️: The aim of this post is not to achieve the best possible results, as this would require considerable dedication in tasks such as Feature Engineering. What is intended is to show a working methodology when using the Modeltime package for global models.
For this post we are going to use the dataset found in the page https://datos.gob.es/ uploaded by the Generalitat de catalunya which measures the electricity generation (GWh) in the period 2005-2020 and whose analysis will be performed for different sectors of the economy.
First, let’s load the necessary 📦 packages and read the data:
The next step is to clean the data. To do this we select those columns we are interested in and transform the date field to adapt it to the format we are interested in modeling.
If you notice, our dataset is not yet in the right format to be used by a global model, we need to put each time series on top of each other (panel data). Let’s get to it:
The next step will be to visualize our time series through the plot_time_series() function. We will also use the plot_anomaly_diagnostics() function to visualize the outliers detected by this function in each series. For the visualization we will make use of the automagic_tabs2 functionality to be able to visualize each time series in a different tab in a comfortable way.
We are going to design a function that adds a flag variable for each column indicating whether the record is an outlier or not (basically we will transform the previous graph into numeric information that the global model can understand). The idea is to create two preprocessing containers, one with the outlier features and one without these features and see with which recipe the global model gets better results:
Once we have the dataset in the format we are interested in, the next step is to continue with the modeling.
Split generation 📊
Our objective will be to predict the next six months of electricity generation for each of the categories (id) that we have in our dataset. To do this, we will follow a split strategy in which we will have our training set and our test set (it will be on the latter that modeltime will perform the calibrations and from which it can calculate the confidence intervals, for example).
First of all, we are going to look for a baseline model that will be our model to beat. Modeltime provides us with the window_reg and naive_reg models precisely for this purpose. The first method allows us to compute a function on a given window (this function is typically the mean, mode or some heavy mean, although the system is flexible enough to allow us to enter any function the user wishes). The second method uses the last observation as if it were the next value in the series, although a seasonal version of this same method is also available. Modeltime also integrates the workflowsets package developed by the Tidymodels team through the modeltime_fit_workflowset() function. The main idea here is to combine many preprocessing containers with different models to evaluate them in a single object in an integrated way. This will be what we will do to try to find our best possible baseline model. First, we will define four preprocessing recipes:
recipe_basic <- recipe(value ~ date + id, data = training(splits))
recipe_basic_features <- recipe(value ~ id + date, data = training(splits)) %>%
step_timeseries_signature(date) %>%
step_rm(matches("(xts$)|(iso$)|(^.pm)")) %>%
step_zv(all_predictors()) %>%
step_dummy(all_nominal_predictors())
recipe_outliers <- recipe(value ~ ., data = training(splits))
recipe_outliers_features <- recipe(value ~ ., data = training(splits)) %>%
step_timeseries_signature(date) %>%
step_rm(matches("(xts$)|(iso$)|(^.pm)")) %>%
step_zv(all_predictors()) %>%
step_dummy(all_nominal_predictors())
Next, we generate different model specifications by varying the desired arguments. In the case of baseline models, we will vary the window length for the window_reg algorithm and the seasonal period for the naive_reg algorithm. This will allow us to generate multiple specifications that will later be combined with the preprocessing recipes generating all possible combinations of preprocessing + model. To generate the specifications by varying the argument, we will use the function implemented in modeltimecreate_model_grid():
Once we have our models, the next step is to cross them with the four preprocessing recipes to create the workflowsets object. The models are then calculated through the modeltime_fit_workflowsets() function (new functionality that allows integrating the Modeltime and Workflowsets package). We will do it in parallel so that the result takes less time to execute using six cores:
Next, we are going to select the best model for each of the sectors of the economy based on the rmse metric. Notice that we are obtaining the metrics locally thanks to the acc_by_id = TRUE argument of the modeltime_accuracy() function. This will allow us to get the metrics for each time series within each trained model. Note that for this option to work correctly it is necessary to pass earlier in the modeltime_calibrate() function the argument id = "id". This functionality has been introduced by Matt Dancho in the latest version of Modeltime (0.7.0):
Notice that we can also calculate the metrics globally (one for each model) which is what was available until this latest version in Modeltime. Let’s see how we could do it:
We are going to cross against the table that we had generated with the calculated models to keep only the ones we are interested in and thus generate a reduced modeltime_table object. Notice that in the data_forecasted object we are going to have the predictions of the 13 models for each and every one of the time series, so we will have to filter for each model the prediction we are interested in based on the previous table (otherwise, for each time series the predictions of all the available models would be painted):
Next we are going to paint the predictions of each model with their confidence intervals. To do this, we will create a specific function to take care of this task (we could use the plot_modeltime_forecast() function, but since we have so many series, it is not quite suitable for visualization in this report and we want to use the automagic_tabs to generate a tab for each series):
Once we have the baseline models, it’s time to move on and try to beat them. In this post we will try to do it using the XGBoost algorithms and the Prophet + Catboost model from the boostime package. Let’s start by finding the best parameters for the XGBoost algorithm. In this case, we will not use the worfklowsets functionality of combining multiple preprocessing steps, but we will create a workflow consisting of a container and a model and search directly for its best hyperparameters (this will save us a lot of computation time in this example and we will see another way to proceed). It is also worth noting the fact that the resamples computed for the XGBoost model and the Prophet+Catboost model are done differently. The explanation is that the latter is a sequential model while the former does not have this feature.
Next we do the same for Prophet + Catboost. First we generate the resamples that we will need to find the best hyperparameters. Although the data looks a bit chaotic, this is because all the series are contained, the important thing is to see where the training and testing subsets start and end in each of the partitions. If you want to see the data of each of the series in each of the partitions, you can consult this post where we do it.
We now generate the recipe and specification of the Prophet + Catboost model through the boostime package. We will use a multiplicative model because some of the series we have encountered have this behavior so it is possible that this model behaves better. Finally, we will use the tune_grid() function to launch the possible configurations on the resamples and see what are the optimal values for the parameters. The tune_grid function will use by default the RMSE and RSQ metrics to estimate the error, but this can be changed using yardstick::metric_set(), but we will talk about this in another post. As this process can be somewhat cumbersome, we will enable the parallel computing option through the control_grid() function with the argument allow_par = TRUE. If you want to learn more about this functionality, you can read this article I wrote with Matt Dancho on the subject.
Below we show the metrics of each global model broken down by time series. As mentioned above, this is a new addition to the latest version of Modeltime and to achieve this it is necessary to include the id argument in the modeltime_calibrate() function and then pass the acc_by_id = TRUE argument in the modeltime_accuracy() function:
Finally, we are going to see which are the best models for each of the time series in order to see if they are better than the baseline models we had selected at the beginning:
It can be clearly seen that the baseline models achieve better metrics than these second models we have trained. Although it may seem surprising, simple models can actually give acceptable results when the parameters are tuned (even better than a priori more complex models). Moreover, as we said at the beginning of the post, a search for better variables (feature engineering) could improve the results of the XGBoost and Prophet + Catboost models.
